Accelerating inexact successive quadratic approximation for regularized optimization through manifold identification

Abstract For regularized optimization that minimizes the sum of a smooth term and regularizer promotes structured solutions, inexact proximal-Newton-type methods, or successive quadratic approximation (SQA) are widely used for their superlinear convergence in terms iterations. However, unlike counter parts optimization, they suffer from lengthy running time solving subproblems because even approximate solutions cannot be computed easily, so empirical cost is not as impressive. In this work, we first show partly regularizers, although general identify active manifold makes objective function smooth, generated by commonly-used subproblem solvers will manifold, with arbitrarily low solution precision. We then utilize property to propose an improved SQA method, $$^{+}$$ + , switches efficient methods after identified. wide class degenerate possesses only iterations, but also per iteration bounded. particular, our result holds on problems satisfying sharpness condition more than existing literature. prove iterate under SQA, which novel family could easily violate classical relative-error frequently proving similar conditions. Experiments real-world support improves over some modern optimization.